Method, apparatus, and computer-readable storage medium for performing a braking operation of a vehicle

ABSTRACT

Method, apparatus, and computer-readable medium for performing a braking operation of a vehicle when an object is detected in front of the vehicle. The method includes acquiring at least one image of an external environment of the vehicle, determining a road condition of a road of the external environment of the vehicle based on the acquired at least one image, obtaining, based on the determined road condition and from memory, a braking table of one or more braking tables including distances and corresponding vehicle speeds at which the braking operation is performed, acquiring a speed of the vehicle and a distance between a preceding object and the vehicle, comparing the acquired speed of the vehicle and the acquired distance between the preceding object and the vehicle to the braking table, and sending, based on the comparison, an instruction to perform the braking operation of the vehicle.

BACKGROUND Field of the Disclosure

The present disclosure relates to performing a braking operation of avehicle in response to detection of a preceding obstacle.

Description of the Related Art

Adverse weather has major impacts on the safety and operations of allroads, from signalized arterials to Interstate highways. Weather affectsdriver behavior, vehicle performance, pavement friction, and roadwayinfrastructure, thereby increasing the risk of crashes. For instance,rain, snow, and ice may dramatically affect the ability of a driver tosafely operate a vehicle as driver vision, decision making, and vehiclehandling are impaired.

To date, a robust approach to performing a braking operation of avehicle during adverse weather conditions has yet to be developed.

The foregoing “Background” description is for the purpose of generallypresenting the context of the disclosure. Work of the inventors, to theextent it is described in this background section, as well as aspects ofthe description which may not otherwise qualify as prior art at the timeof filing, are neither expressly or impliedly admitted as prior artagainst the present invention.

SUMMARY

The present disclosure relates to a method, apparatus, andcomputer-readable storage medium comprising processing circuitryconfigured to perform a method of a braking operation of a vehicle.

According to an embodiment, the present disclosure further relates amethod of a braking operation of a vehicle, comprising acquiring atleast one image of an external environment of the vehicle, determining aroad condition of a road of the external environment of the vehiclebased on the acquired at least one image, obtaining, based on thedetermined road condition and from memory, a braking table of one ormore braking tables including distances and corresponding vehicle speedsat which the braking operation is performed, acquiring a speed of thevehicle and a distance between a preceding object and the vehicle,comparing the acquired speed of the vehicle and the acquired distancebetween the preceding object and the vehicle to the braking table, andsending, by processing circuitry and based on the comparison, aninstruction to perform the braking operation of the vehicle.

According to an embodiment, the present disclosure further relates to anapparatus for providing a braking operation of a vehicle, comprisingprocessing circuitry configured to acquire at least one image of anexternal environment of the vehicle, determine a road condition of aroad of the external environment of the vehicle based on the acquired atleast one image, obtain, based on the determined road condition and frommemory, a braking table of one or more braking tables includingdistances and corresponding vehicle speeds at which the brakingoperation is performed, acquire a speed of the vehicle and a distancebetween a preceding object and the vehicle, compare the acquired speedof the vehicle and the acquired distance between the preceding objectand the vehicle to the braking table, and send, based on the comparison,an instruction to perform the braking operation of the vehicle.

According to an embodiment, the present disclosure further relates to anon-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method of a braking operation of a vehicle,the method comprising acquiring at least one image of an externalenvironment of the vehicle, determining a road condition of a road ofthe external environment of the vehicle based on the acquired at leastone image, obtaining, based on the determined road condition and frommemory, a braking table of one or more braking tables includingdistances and corresponding vehicle speeds at which the brakingoperation is performed, acquiring a speed of the vehicle and a distancebetween a preceding object and the vehicle, comparing the acquired speedof the vehicle and the acquired distance between the preceding objectand the vehicle to the braking table, and sending, by processingcircuitry and based on the comparison, an instruction to perform thebraking operation of the vehicle.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the followingclaims. The described embodiments, together with further advantages,will be best understood by reference to the following detaileddescription taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is a graphical illustration of road friction under varyingweather conditions;

FIG. 2 is an illustration of a vehicle, according to an exemplaryembodiment of the present disclosure;

FIG. 3 is a flow diagram of a method of performing a braking operationof a vehicle, according to an exemplary embodiment of the presentdisclosure;

FIG. 4 is a flow diagram of a sub process of a method of performing abraking operation of a vehicle, according to an exemplary embodiment ofthe present disclosure;

FIG. 5 is a flow diagram of a sub process of a method of performing abraking operation of a vehicle, according to an exemplary embodiment ofthe present disclosure;

FIG. 6 is a graphical illustration of a braking table, according to anexemplary embodiment of the present disclosure;

FIG. 7A is a flow diagram of a sub process of a method of performing abraking operation of a vehicle, according to an exemplary embodiment ofthe present disclosure;

FIG. 7B is a flow diagram of a sub process of a method of performing abraking operation of a vehicle, according to an exemplary embodiment ofthe present disclosure;

FIG. 7C is a flow diagram of a sub process of a method of performing abraking operation of a vehicle, according to an exemplary embodiment ofthe present disclosure;

FIG. 8 is a schematic illustrating the communication architecture of asystem including a vehicle wherein processing is performed remotely,according to an exemplary embodiment of the present disclosure; and

FIG. 9 is a block diagram of a vehicle control system, according to anexemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). Reference throughoutthis document to “one embodiment”, “certain embodiments”, “anembodiment”, “an implementation”, “an example” or similar terms meansthat a particular feature, structure, or characteristic described inconnection with the embodiment is included in at least one embodiment ofthe present disclosure. Thus, the appearances of such phrases or invarious places throughout this specification are not necessarily allreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics may be combined in any suitable manner inone or more embodiments without limitation.

Performing a braking operation in adverse conditions, and in response toa roadway obstacle, can be difficult for human drivers and autonomousvehicles, alike. Braking systems, including emergency braking systems,use information provided by a variety of vehicle sensors, includingultrasonic sensors, cameras, radar, and laser scanners, in order toobserve the environment, create a map of the environment, and detectobstacles such as vehicles or other objects that are present within adriving tube of a host vehicle. Based on information from one or more ofthe variety of vehicle sensors indicating a position of an obstacle, adistance to collision can be calculated. This calculated distance canthen be evaluated in view of a speed of the vehicle to determine if abraking operation is required in order to avoid imminent collision.

The widespread adoption of vehicles having at least a requisite level ofautonomy offers the possibility of improved navigation during adverseweather conditions. Relatively limited attention, however, has beendirected to enhancing performance of semi- or fully-autonomous vehiclesunder such conditions. Notably absent is a consideration of roadwayconditions which can greatly impact the ability of the vehicle toperform the braking operation and to stop the vehicle prior tocollision. For example, adverse weather conditions, including winterweather conditions like rain, snow, and ice, may dramatically affect adistance required by a vehicle in order to safely perform a brakingoperation and stop the vehicle. When driving on ice, the distancerequired to safely perform a braking operation can be increased by asmuch as 10 times that of a braking operation performed during dryweather conditions.

This is largely due to the fact that tire traction may be severelyreduced in such conditions, as evidenced by the graphic of FIG. 1. InFIG. 1, where slip ratio is on the x-axis and friction coefficient is onthe y-axis, it can be appreciated that varying road types and varyingroad conditions have a significant impact on traction between a vehicleand a road surface when the vehicle is traveling at a constant speed.

It can be intuited that a maximal friction coefficient is advantageousfor performing a braking operation. However, for different roadconditions, the maximal friction coefficient varies widely and may occurat different slip ratios, as shown in FIG. 1. As a result, increasedstopping distances during braking are required in order to account forreduced friction coefficients and increased slip ratios.

In an effort to develop a generalized approach to braking operations,previous work has established a ‘preferred range’, an example of whichis indicated by the shaded area of FIG. 1. Such a ‘preferred range’establishes a range of friction coefficient-slip ratio values, informedby empirical data on varied road surface conditions, which can begenerally implemented within present day vehicle braking operationsindependent of actual road types and road conditions. As can be observedin FIG. 1, however, this approach fails to exploit the maximal frictioncoefficient of each vehicle and, potentially, results in brakingoperations that are not appropriate for the variety of road surfaceconditions that may be experienced in real life.

Thus, it can be deduced from the above that any braking operationperformed during adverse weather conditions will need to consider adynamic set of parameters, including a type and a quality of a roadsurface, in order to safely stop. Instead of applying a generalizedapproach based only on vehicle traits (e.g., weight, tire size, tirewidth), a braking operation will need to consider the road type and theroad condition, along with other parameters related to the specificvehicle and roadway involved, in order to safely preform a brakingoperation. For instance, the keys to safely handling a vehicle onslippery winter roads include slower speeds, gentler stops and turns,and increased following distances. Such considerations must be accountedfor when performing a braking operation in a semi- or fully-autonomousvehicle.

In view of the above, it can be appreciated that any braking operationneeds to consider roadway conditions in order to determine how and whento apply a braking operation.

Accordingly, the present disclosure describes a method, apparatus, andcomputer-readable storage medium for performing a braking operation of avehicle during adverse weather conditions.

According to an embodiment, the present disclosure takes into accountroad conditions, including road surface type and road surface quality,in calculating a distance to collision with an obstacle within a drivingtube of a vehicle.

In an embodiment, the road surface type and the road surface quality canbe detected via one or more of a plurality of vehicle sensors, includingcameras. In certain instances, only one camera may be sufficient. Inother instances, using more than one camera may increase confidence in aroad type classification and/or road quality classification. Moreover,more than one camera may aid in a distance determination.

In an embodiment, the one or more of a plurality of vehicle sensors maybe a camera(s) used to acquire images of an external environment of thevehicle such that the road surface type and road surface quality can bedetermined. This determination can include classification of the roadsurface type as asphalt, gravel, cobblestone, dirt, rocks, and the like,and classification of the road surface quality as icy, wet, dry, snowy,and the like, as appropriate.

According to an embodiment, the acquired images can be processedaccording to an algorithm based on an artificial neural network or othercomputer vision algorithm for image segmentation and imageclassification. The processed images can be used to provide a brakingsystem of a vehicle with information regarding road surface type and/orroad surface quality. In an example, the artificial neural network maybe a convolutional neural network. The convolutional neural network maybe trained with labeled road images to be able to automatically learnthe features that are most beneficial to road condition classification.

In an embodiment, the acquired images may be first processed to detect aroad and, second, to classify the detected road. In an example, semanticsegmentation may be applied to the acquired image to detect and localizethe road within the acquired image.

According to an embodiment, braking operations of a vehicle may be basedon one or more parameters, including a distance to collision which maybe calculated on the basis of one or more of a plurality of vehiclesensors, and/or a time to collision which may be calculated on the basisof the distance to collision and a relative speed of the vehicle. Forinstance, if the time to collision or the distance to collisionsatisfies a certain threshold, a braking operation may be initiated.

According to an embodiment of the present disclosure, differentthresholds can be defined by different braking tables that correspond toa road surface type, road surface quality, and certain vehicle traits.In this way, different braking tables can be generated according to theroad type and the road surface detected using a computer visionalgorithm. Such braking tables may include, among others (a) a brakingtable for ideal conditions (e.g., dry asphalt and concrete), (b) abraking table for ice (e.g., icy asphalt and concrete), (c) a brakingtable for heavy rain (e.g., wet asphalt and concrete), and (d) a brakingtable for snow (e.g., snowy asphalt and concrete).

Moreover, the different braking tables can be calculated based onexperimental tests performed on each of a variety of scenarios or basedon a braking algorithm that uses slip ratio, as in FIG. 1, and frictioncoefficient, together with a braking table for ideal conditions, inorder to determine the different braking tables. For example, the (b)braking table for ice could use values from (a) adapted based on theinputs received from the algorithm. In other words, consider a hostvehicle driving on an icy asphalt road at 10 km/h towards a staticobstacle. Under ideal conditions, the host vehicle may initiate abraking operation at a distance of 2.1 m from the obstacle. However, onthe icy asphalt road, additional stopping distance is needed.Accordingly, an output of a braking algorithm may be 4 and a brakingoperation may be triggered when the host vehicle is 8.4 m away from thestatic obstacle (instead of 2.1 m away from the obstacle).

Further to the above, and as it is not good practice to perform a fullbraking operation (i.e. emergency braking operation) when driving onice, snow, or water, certain embodiments of the present disclosuredescribe performance of a pre-fill braking operation in advance of afull braking operation. Pre-fill braking is a braking function thatprepares the brakes for a hard braking operation by automaticallyincreasing pressure in the brake system before an emergency brakingoperation is applied. By ‘prefilling’ the brake hydraulics with fluid,braking system reaction time can be shortened, thereby leading to ashorter braking distance and safer braking operation. To this end, thepre-fill braking operation also brings the brake pads closer to thebraking discs, thereby speeding up the braking effect for optimalstopping. In such embodiments, a corresponding braking table includesdata related to a pre-fill braking operation, and the pre-fill brakingoperation can be initiated when certain pre-fill braking operationconditions are met. During a pre-fill braking operation, the acceleratorpedal may be deactivated.

In an embodiment, and in the absence of sensors configured to locallydetermine these parameters, the methods described herein may furtherinclude use of geolocation-based weather data from the Internet toestimate road surface type and road surface quality. Methods describedherein may also use a road surface database including annotated roadwaymaps to understand, upon querying the road surface database based ongeolocation data, for instance, a type of road surface the vehicle istraveling on (e.g., asphalt, cement). Each of the annotated roadway mapsmay be a high definition (HD) map.

It can be appreciated that the methods introduced above can beimplemented within a vehicle implementing an active safety system,within a semi-autonomous vehicle (SAV), and/or within a fully-autonomousvehicle. With reference again to the Figures, FIG. 2 is an illustrationof an SAV, according to an exemplary embodiment of the presentdisclosure. In order to operate accurately and with precision, the SAV200 can be outfitted with a plurality of vehicle sensors 205, including,among others, one or more cameras 206, one or more surround view cameras207, at least one radar (radio detection and ranging; herein “radar”)208, at least one LiDAR (light detection and ranging; herein “lidar”)209, at least one ultrasonic sensor 210, and one or more corner radar211. Data acquired from the plurality of vehicle sensors 205 can be sentto a vehicle control system 201, comprising, among other components,processing circuitry(s), a storage medium, image processingcircuitry(s), and communication circuitry(s), in order to be processed,locally and/or globally, and utilized in vehicle operation. In oneembodiment, the vehicle control system 201 can be an electronic controlunit, “electronic control unit” being used herein to describe anyembedded system in automotive electronics that controls one or moreelectrical systems or subsystems in a vehicle, including, among others,a telematics control unit, an engine control module, and a powertraincontrol module. One implementation of the vehicle control system 201 isillustrated in FIG. 9. The above-described plurality of vehicle sensors205 of the SAV 200 will be discussed in brief below.

Regarding the one or more cameras 206, the cameras may be positionedalong a forward panel of the SAV 200 and arranged such that, in the caseof a plurality of cameras, a parallax is created between the viewpoints.The parallax can be subsequently exploited, based upon the fixedgeometric relationship between the viewpoints along the panel of the SAV200, to determine a distance to an obstacle, or impediment. To this end,the one or more cameras 206 may provide mono- or stereo-scopicperspective. The one or more cameras 206 can employ, among othersensors, CMOS image sensors.

Regarding the one or more surround view cameras 207, the surround viewcameras may be positioned around the SAV 200 in order to create aparallax and to obtain a 360° representation of the vehiclesurroundings. As before, the parallax can be subsequently exploited,based upon the fixed geometric relationship between the viewpoints, inorder to determine a distance to an obstacle, or impediment. The one ormore surround view cameras 207 can employ, among other sensors, CMOSimage sensors.

Regarding the above-described one or more cameras 206 and one or moresurround view cameras 207, in addition to distancing, the output of thecameras 206, 207 can be further processed by the vehicle control system201 to identify the vehicle surroundings. For instance, the imageprocessing circuitry(s) of the vehicle control system 201 can performone or more image classification operations on an output of the cameras206, 207 in order to determine a road condition (e.g., a road surfacetype and a road surface condition).

Regarding the at least one radar 208, the radar may be positioned alonga forward panel of the SAV 200. The at least one radar 208 can be oneselected from a group of radars including, among others, short rangeradar, medium range radar, and long range radar. In an embodiment, andas employed commonly in Adaptive Cruise Control and Automatic EmergencyBraking Systems, the at least one radar 208 may be a long range radarwith an operational range of, for example, a few hundred meters. The atleast one radar 208 may be used to measure a distance between the SAV200 and a preceding obstacle, or impediment.

Regarding the at least one lidar 209, the lidar may be positioned, forexample, at a forward facing position and/or at a position with a 360°viewpoint. The at least one lidar 209 can be an infrared lidar systemusing a rotating laser via a micro-electro-mechanical system, asolid-state lidar, or any other type of lidar. In one embodiment, the atleast one lidar 209 can provide a 905 nm wavelength with up to a 300meter operational range.

In an embodiment, radar and lidar may be interchangeable, mutatismutandis, for certain distancing applications.

Regarding the at least one ultrasonic sensor 210, the ultrasonic sensormay be disposed at corners of the SAV 200 for, in particular,short-range distancing. The at least one ultrasonic sensor 210 can be anultrasonic sensor having asymmetric directivity (110°×50°), shortringing time and high sound pressure, sensitivity and reliability, andbe configured to produce, among others, a 40 kHz, 48 kHz, 58 kHz, or 68kHz nominal frequency as required by the current situation.

Regarding the one or more corner radars 211, the radars can besubstantially similar to the above-described at least one radar 208.Deployed as corner radars, the one or more corner radars 211 can beshort range radar or medium range radar, as demanded, and can bebroadband Frequency Modulated Continuous Wave radar.

In an embodiment, a combination of longitudinally-acquired (time-based)data from the above-described camera and distancing systems (radarand/or lidar, front cameras, ultrasonic) can be used to extract speedand outlines of obstacles and moving objects.

In an embodiment, the sensors exploited in performing distancing can bebased on a traveling speed of the SAV. For instance, surround-viewcameras and ultrasonic sensors may be used when the SAV is traveling atspeeds below ˜15 km/h, while camera, radar, and lidar may be used whenthe SAV is traveling at speeds above 50 km/h. A combination of sensorsmay be used when the SAV is traveling at speeds therebetween.

Of course, it can be appreciated by one of ordinary skill in the artthat the above-described plurality of sensors 205 do not constitute anexhaustive list and are merely exemplary of vehicle sensors that may befound on an SAV or other vehicle. In that context, any combination ofvehicle sensors, described herein or not, can be integrated in order toachieve the function of the methods described herein.

According to an embodiment, and with reference to FIG. 3, theabove-described vehicle sensors, in communication with the vehiclecontrol system 201 of FIG. 2, allow for implementation of a brakingoperation based on, among other relevant factors, a road surface type,road surface quality, and a distance to a preceding obstacle. Forexample, the one or more cameras 206 can be used to obtain images of anenvironment surrounding the SAV 200. The obtained images can beprocessed by applying a neural network trained to detect and classify aroad condition of a road of the obtained image, the classificationinforming a selection of a corresponding braking table. Further, the atleast one radar 208 and/or the at least one lidar 209 can be used todetermine a distance to a preceding object, or obstacle, that can beused in conjunction with the corresponding braking table to determine ifand when a braking operation needs to be performed.

Method 320 of FIG. 3, which may be controlled by the vehicle controlsystem of the SAV, sets forth a process for determining if a brakingoperation needs to be performed based on a road condition-specificbraking table, a traveling speed of an SAV, and a distance between theSAV and a preceding obstacle. It should be appreciated that an SAV ismerely exemplary of a vehicle having at least some level of autonomythat would allow for the necessary processes described herein to beperformed and is not meant to be exclusive. Moreover, it should beappreciated that the preceding obstacle may be any impediment to vehicletravel, such as a vehicle, a tree, an animal, and the like.

According to an embodiment, method 320 may be initiated by detection ofa preceding obstacle at step 315 of method 320. In one instance,detection can be defined as a function of a calculated distance betweenthe SAV and the forward positioned object, or preceding obstacle. Whenthe calculated distance between the SAV and the forward positionedobject is below a predetermined distance, the forward positioned objectis determined to be a detected preceding obstacle. In another instance,detection can be defined as a function of a confidence level of aclassification of an acquired image of the surrounding environment. Ineither instance described above, or following detection by anothermeans, method 320 may continue following detection at step 315 at method320.

It can be appreciated, however, that detection of a preceding obstacle,in an embodiment, is not necessary for performance of method 320, as thepresence of any specific type of object is irrelevant to the processesof method 320. Accordingly, the remainder of method 320 will bedescribed without requirement for preceding obstacle detection.

At step 325 of method 320, an image of an external environment of theSAV may be acquired via one or more cameras of the SAV. The acquiredimage may include a driving tube of the SAV, for instance. In anexample, the one or more cameras of the SAV may be one camera. At subprocess 330 of method 320, a road condition may be determined on thebasis of the acquired image of the external environment of the SAV.Described in detail with reference to FIG. 4, the determination mayinclude, in an example, detection of the road from the acquired imageand determination of a road condition thereof. The determined roadcondition can include a determined road surface type and a road surfacequality that can be used at sub process 340 of method 320 to obtain acorresponding braking table. The corresponding braking table may be oneof a plurality of reference braking tables that provide guidelines bywhich a braking operation (e.g., emergency braking operation, pre-fillbraking operation) may be initiated. Sub process 340 of method 320 willbe described in greater detail with reference to FIG. 5.

Concurrently to the above, host vehicle metrics may be obtained. Forinstance, a current speed of the SAV may be acquired from the vehiclecontrol system at step 345 of method 320 and a distance between the SAVand a preceding obstacle may be acquired at step 346 of method 320. Itcan be appreciated that each of the SAV speed and the distance to thepreceding obstacle may be continuously and iteratively determined inorder to provide real-time information for processing at sub process 355of method 320. In an example, step 346 of method 320 may deploy the atleast one radar and/or the at least one lidar of the SAV, among othertools, for the determination of the distance to the preceding obstacle.To this end, a time-dependence of a signal from the at least one radar,for example, may be used to calculate the distance to the precedingobstacle.

Outputs from the above-described steps of method 320 can be received forprocessing at sub process 355 of method 320. Sub process 355, which willbe described in greater detail with respect to FIG. 7A through FIG. 7C,describes a determination of whether the acquired SAV speed and theacquired distance to the preceding obstacle satisfy conditions to begina braking operation. Such a determination can be made in view of thecorresponding braking table obtained at sub process 340 of method 320.In an example, the braking operation may be a full braking operation, oremergency braking operation. In an example, the braking operation mayinclude a pre-fill braking operation and an emergency braking operation.Such braking operations will be described in greater detail withreference to subsequent Figures, but it can be appreciated that eitherbraking operation may be initiated based on a determination that the SAVspeed and the distance to the preceding obstacle satisfy conditions ofthe road surface condition-based braking table.

In view of the above, it can be appreciated that once it is determinedthat a braking operation is required the vehicle control system maymodulate the speed and braking operation of the SAV in order to satisfythe corresponding braking table. Such modulation may include sending acontrol signal to the powertrain control module in order to modulate thespeed and braking systems of the vehicle.

Of course, while method 320 is described above as implemented in asingle instance, the steps of method 320 may be performed iteratively inorder to update controlling signals that modulate performance of theSAV. For instance, the SAV speed and the distance to the precedingobstacle can be iteratively determined in real-time so that, in theevent the distance to the preceding obstacle abruptly changes and isgreater than that required for safe braking, the speed and brakingoperation of the SAV may be adjusted, appropriately. Moreover, thecondition of the road can be iteratively determined so that a dryingroad surface can be properly reflected in selection and acquisition of acorresponding braking table.

With reference now to FIG. 4, sub process 330 of method 320 will bedescribed. At step 431 of sub process 330, an image acquired of theexternal environment of the SAV at step 325 of method 320 can beprocessed in order to identify and isolate a road of the acquired image.

To this end, a number of image processing techniques can be applied,including image segmentation. In an example, semantic image segmentationis used, allowing for detection and localization of a road within theacquired image. The semantic image segmentation may be performed byapplying a convolutional neural network to the acquired image, theconvolutional neural network having been trained to label specificregions of the acquired image according to known categories viapixel-based classification. In other words, semantic image segmentationpermits labeling of each pixel of the acquired image with acorresponding class of what is being represented (e.g., road, car, tree,pedestrian, animal, etc.).

In an embodiment, the convolutional neural network described above canbe obtained by transfer learning. In other words, the convolutionalneural network may be an existing, well-studied image classificationnetwork that serves as an encoder module for the network, appending adecoder module with transpose convolutional layers to upsample coarsefeature maps into a full-resolution segmentation map. Accordingly, thepresent disclosure may include a fully convolutional-adaptation ofAlexNet, the VGG net, and GoogLeNet.

At step 432 of sub process 330, the segmented road of the acquired imagemay be further processed to determine a road condition thereof. The roadcondition may be a road surface quality including wet, dry, and thelike, and/or a road surface type including gravel, asphalt, concrete,dirt, and the like. The road surface quality may further includeclassifications regarding a type of wetness, such as wetness resultingfrom snow, ice, rain, or other type of moisture.

The further image processing of the segmented road may be by applicationof an image classification technique. The image classification approachmay be based on application of a convolutional neural network to thesegmented road of the acquired image, the convolutional neural networkhaving been trained to label the entire image, or segmented road of theacquired image, as one of a number of classes. In other words, imageclassification is the process of taking an input and outputting a classor a probability that the input is a particular class. Considered inview of the present disclosure, the class that may be predicted may be aroad surface quality and road surface type such as, among others, ‘wet,snow, asphalt’, ‘wet, ice, concrete’, and ‘dry, asphalt’. In this way,the segmented road may be further processed to generate a classificationor a probability that the segmented road is one of a number of classes.

While sub process 330 of method 320 is described above as including atwo-step process, wherein a road is first segmented from the acquiredimage and then further classified into a road condition class, subprocess 330 of method 320 may be alternatively performed within a singleclassification process. For instance, the semantic image segmentationtechnique described above may be configured to, instead of labeling aregion of the acquired image as road, label a region of the acquiredimage as ‘dry, concrete’ road. It can be, of course, appreciated thatsuch approach may require more intensive training and may require,during run time, increased processing power.

In either instance, a resulting classification of sub process 330 ofmethod 320 may be output to sub process 340 of method 320 and used forobtaining an appropriate braking table.

In addition to the above, the determination of the road condition may bebased on current weather conditions, information from neighboringvehicles obtained via vehicle-to-vehicle communication, and/or storedinformation related to the currently traveled road. For instance, theSAV may be configured to receive a weather forecast from ameteorological service based on a current geolocation of the SAV, andthe weather forecast may be used in order to determine the road surfacequality. In another instance, the SAV may be configured to access a HDmap of the currently traveled road, annotations of the HD map indicatinga road surface type that the SAV is currently traveling on. The HD mapmay then be used to determine a current road surface type that can beused in conjunction with the weather based-road surface quality todetermine a proper braking table.

Turning now to FIG. 5, the determined road condition of sub process 330of method 320 can be used in order to obtain a corresponding brakingtable. The corresponding braking table may include a current vehiclespeed and at least one set of minimum distances indicating when eitheror both of a pre-fill braking operation and an emergency brakingoperation should be initiated.

At step 541 of sub process 340, current SAV parameters may be acquiredin combination with the determined road condition from step 325 of subprocess 330. The current SAV parameters can include a vehicle make and avehicle model, appreciating that certain SAV parameters, such as weight,will impact an ability of the SAV to be stopped. Other current vehicleparameters of interest can include tire size, tire width, and the like,and may be obtained according to OEM components of the vehicle make andvehicle model or from information obtained by querying an operator ofthe SAV following installation of requisite components.

At step 543 of sub process 340, the acquired SAV parameters and roadcondition determination can be used to obtain a braking table from adatabase 542 of reference braking tables. The database may be searchableby the above-described road condition determination as well as theacquired SAV parameters. Upon comparison of the acquired SAV parametersand the determined road condition with a reference braking table, areference braking table determined to be highly correlated may be outputfrom sub process 340 as the braking table to be implemented within theSAV. In an embodiment, the highly correlated reference braking table maybe one having an exact match between acquired SAV parameters and thedetermined road condition. In another embodiment, the highly correlatedreference braking table may be one having a greatest match of factorsbetween acquired SAV parameters and the determined road condition.

Each of the reference braking tables can include, in a first column, aspeed of the current vehicle and, in at least one subsequent column, aminimum distance to a preceding obstacle for a given road condition, theminimum distance to the preceding obstacle being a distance, associatedwith a given speed and a respective braking operation (e.g., pre-fillbraking operation, emergency braking operation), at which the respectivebraking operation should be triggered. Alternatively, each of thereference braking tables can include, in a first column, a speed of thecurrent vehicle and, in at least one subsequent column, a time tocollision with a preceding obstacle for a given road condition, the timeto collision being a time, associated with a given speed, at which abraking operation should be triggered. Though it will be appreciatedthat both approaches can be implemented, mutatis mutandis, the remainingdisclosure will focus on a braking table that includes a current vehiclespeed and a minimum distance to a preceding obstacle, for simplicity.Reference tables will be discussed in greater detail with reference toFIG. 6 and subsequent Figures.

In an embodiment, each braking table of the database 542 can becalculated based on experimental tests performed for each scenario ofSAV parameters and road conditions. In another embodiment, each brakingtable of the database 542 may be calculated based on slip ratios andfriction coefficients in view of a braking table established for anideal road condition. For instance, a braking table for an icy conditionmay be based on a braking table for a dry road condition but adaptedbased on slip ratios and friction coefficients calculated in real time.In this way, a braking table may be, for instance, a calculated multipleof that for a dry road condition.

An exemplary braking table is shown in FIG. 6. The braking table mayinclude, as in FIG. 6, three or more columns including, in a firstcolumn, a speed of a current vehicle, and, in subsequent columns, aminimum distance to a preceding obstacle. The subsequent columnsdescribing the minimum distance to the preceding obstacle can bespecific to a braking operation and/or to a road condition. As describedabove, the minimum distance values in the braking table indicate aminimum distance between the current vehicle (e.g., the SAV) and apreceding obstacle (e.g., a threshold) by which a braking operationshould be initiated.

As an exemplary implementation of the exemplary braking table of FIG. 6,an SAV traveling at a constant velocity, or constant speed, of 6 km/h,may be considered. In a first example, methods of the present disclosuremay be performed to determine that the road condition is a dry road.Accordingly, ‘Dry’ columns of the table of FIG. 6 may be considered. Attime t=1, the SAV may be at a distance of 1400 mm from a precedingobstacle. Upon analysis of the exemplary braking table of FIG. 6, it canbe appreciated that, given the constant speed of 6 km/h and the distanceto the preceding obstacle, neither the minimum distance to initiate apre-fill braking operation nor the minimum distance to initiate anemergency braking operation has been achieved. At time t=2, however, andgiven the SAV is traveling at the constant speed of 6 km/h, thepreceding obstacle may now be at 1330 mm. Upon analysis of the exemplarybraking table of FIG. 6, it can be appreciated that, given the constantspeed of 6 km/h and the distance to the preceding obstacle, the minimumdistance to initiate the pre-fill braking operation has been achieved(i.e., 1330 mm≤1330 mm) while the minimum distance to initiate theemergency braking operation has not been achieved (i.e. 1330 mm≥1000mm). Accordingly, at time t=2, the pre-fill braking operation may beinitiated. Of course, the preceding obstacle may be at a standstill andthe distance to the preceding obstacle at time t=3 may now be determinedto be 1000 mm. As a result, upon analysis of the exemplary braking tableof FIG. 6, it can be determined that, following the application of thepre-fill braking operation, an emergency braking operation may betriggered (i.e., 1000 mm≤1000 mm).

In a second example, the SAV continues traveling at a constant velocity,or constant speed, of 6 km/h, but methods of the present disclosure areperformed to determine that the road condition is a snow-covered road.Accordingly, ‘Snow’ columns of the table of FIG. 6 may be considered. Asin the first example, the SAV may be 1400 mm from the preceding obstacleat time t=1. In contrast to the above-determined ‘Dry’ road condition,analysis of the exemplary braking table of FIG. 6 indicates that, giventhe constant speed of 6 km/h and the distance to the preceding obstacle,both of the pre-fill braking operation and the emergency brakingoperation should have already been initiated, as the minimum distance toinitiate a pre-fill braking operation and the minimum distance toinitiate an emergency braking operation have already been achieved. Itcan be appreciated that the above example is purely illustrative, andthat, when performed in real-time, a pre-fill braking operation and anemergency braking operation may be initiated according to the ‘Snow’columns of the braking table of FIG. 6.

The above-described examples demonstrate the significant differencebetween traveling on ‘Dry’ road conditions and traveling on‘Snow’-covered road conditions. For instance, given a constant speed of12 km/h, a minimum distance required for initiating an emergency brakingoperation on ‘Snow’-covered road conditions is 2.8× higher than when on‘Dry’ road conditions. A similar relationship exists for pre-fillbraking operations.

Moreover, the exemplary braking table of FIG. 6 demonstrates, relativeto respective emergency braking operations, the impact of roadconditions on when pre-fill braking operations need to be initiated. Forinstance, given a constant speed of 4 km/h, a minimum distance requiredfor initiating a pre-fill braking operation is −30% higher than aminimum distance required for initiating an emergency braking operation.When traveling on a ‘Snow’-covered road, however, a minimum distancerequired for initiating a pre-fill braking operation is −50% higher thana minimum distance required for initiating an emergency brakingoperation. This observable difference is evidence of the utility of themethods described herein and of the importance of detecting andaccommodating different road conditions.

Of course, in either of the ‘Dry’ road condition or the ‘Snow’ roadcondition, or any other detected road condition, it is likely that thecurrent speed of the SAV and the current distance to the precedingobstacle may be values that do not exist within the reference brakingtable. For these instances, interpolation may be performed based on afunction defining the reference values such that intermediate,“unknown”, values can be determined and an appropriate braking operationcan be safely implemented.

With reference now to FIG. 7A, a generalized implementation of subprocess 355 of method 320 will be described.

The braking table obtained at step 543 of sub process 340, the currentSAV speed acquired at step 345 of method 320, and the distance to thepreceding obstacle, or preceding object, acquired at step 346 of method320 can be considered together at step 756 of sub process 355. Thecomparison of these inputs at step 756 of sub process 355 allows for adetermination to be made at step 757 regarding initiation of a brakingoperation.

In an embodiment, the comparison at step 756 of sub process 355 caninclude considering the acquired distance to the preceding object andthe acquired current SAV speed as a data point and comparing the datapoint to the obtained braking table. In this way, the determination atstep 757 of sub process 355 may be that, in the event the data pointfails to satisfy conditions for safe stopping (i.e., the acquireddistance is less than or equal to the minimum distance to performing abraking operation), a braking operation can be initiated. Accordingly,instructions can be sent to the powertrain control module, or otherengine control unit, at step 760 of sub process 355 to initiate thebraking operation. Of course, the determination at step 757 of subprocess 355 may be that a braking operation need not be initiated (i.e.,the acquired distance is greater than the minimum distance to performinga braking operation), as the data point satisfies conditions for safestopping. In this event, sub process 355 may return to step 756 and thecomparison of the obtained braking table, the current SAV speed, and theacquired distance to the preceding object can be iteratively performed.

It is important to note that sub process 355 of method 320, and method320, can be iteratively performed based on newly acquired road conditioninformation, SAV speed information, and preceding object distanceinformation.

In one example, newly acquired road condition information maynecessitate a newly obtained braking table corresponding to the new roadcondition information.

In another example, newly acquired SAV speed information may indicatethat a speed of the SAV has decreased more than expected and, therefore,the braking operation does not need to be sustained to completion.

In another example, newly acquired preceding object distance informationmay indicate that the preceding object has moved out of the driving tubeof the vehicle or that a velocity of the preceding object has increasedbeyond that of the SAV. In such cases, the braking operation, ifinitiated, does not need to be continued to completion.

Returning to the Figures, FIG. 7B and FIG. 7C provide additional detailregarding FIG. 7A in the event a pre-fill braking operation is acomponent of a braking operation. First, with reference to FIG. 7B andin view of FIG. 6D, sub process 355 will be described in the context ofa pre-fill breaking operation.

The braking table obtained at step 543 of sub process 340, the currentSAV speed acquired at step 345 of method 320, and the distance to thepreceding object acquired at step 346 of method 320 can be consideredtogether at step 756′ of sub process 355. The comparison of these inputsat step 756′ of sub process 355 allows for a determination to be made atstep 757 regarding initiation of a pre-fill braking operation. Thecomparison may be performed in view of a first condition defining whenthe pre-fill braking operation should be initiated. For instance, apre-fill braking column of the braking table may, in part, define thefirst condition.

To this end, the comparison at step 756′ of sub process 355 can includeconsidering the acquired distance to the preceding object and theacquired current SAV speed as a new data point and comparing the newdata point to the pre-fill braking operation data of the braking table.Such a comparison can be appreciated in view of FIG. 6.

In this way, the determination at step 757 of sub process 355 may bethat, in the event the new data point fails to satisfy conditions forsafe stopping, a pre-fill braking operation can be initiated.Accordingly, instructions can be sent to the powertrain control module,or other engine control unit, at step 760 of sub process 355 to initiatethe pre-fill braking operation. Of course, the determination at step 757of sub process 355 may be that a pre-fill braking operation need not beinitiated, as the new data point satisfies conditions for safe stopping.In this event, sub process 355 may return to step 756′ and thecomparison of the obtained braking table, the current SAV speed, and theacquired distance to the preceding object can be iteratively performed.

The same analysis can be applied to emergency braking operations. Forinstance, and with reference now to FIG. 7C, a subsequent comparison ofthe obtained braking table, the acquired SAV speed, and the precedingobject distance can be made at step 756″ of sub process 355. Thecomparison of these inputs at step 756″ of sub process 355 allows for adetermination to be made at step 757 regarding initiation of anemergency braking operation. The comparison may be performed in view ofa second condition defining when the emergency braking operation shouldbe initiated. For instance, an emergency braking column of the brakingtable may, in part, define the second condition.

To this end, the comparison at step 756″ of sub process 355 can includeconsidering the acquired distance to the preceding object and theacquired current SAV speed as a data point and comparing the data pointto the emergency braking operation data of the braking table. Such acomparison can be appreciated in view of FIG. 6.

In this way, the determination at step 757 of sub process 355 may bethat, in the event the data point fails to satisfy conditions for safestopping or stopping of the SAV following application of the pre-fillbraking operation, an emergency braking operation need be initiated.Accordingly, instructions can be sent to the powertrain control module,or other engine control unit, at step 760 of sub process 355 to initiatethe emergency braking operation. Of course, the determination at step757 of sub process 355 may be that an emergency braking operation neednot be initiated, as the data point satisfies conditions for stopping.In this event, sub process 355 may return to step 756″ and thecomparison of the obtained braking table, the current SAV speed, and theacquired distance to the preceding object can be iteratively performed.

It is important to note that sub process 355 of method 320, and method320, can be iteratively performed in real-time. Moreover, sub process355 of method 320 and method 320 may be iteratively performed based onnewly acquired road condition information, SAV speed information, andpreceding object distance information.

In an example, following comparison of the preceding object distance,the SAV speed, and the obtained braking table at step 756″, sub process355 may return to either step 756′ or step 756″. This allows foraccommodation of drastic changes in the above parameters that mightaffect the application of the braking operation.

In an embodiment, the above-described methods can be implemented onlocal hardware and/or via communication with remote hardware. Imageprocessing tasks may be performed on local processing circuitry of thevehicle control system of the SAV and/or by wireless communication withremote circuitry, such as servers. The reference database of brakingtables may be stored locally by the vehicle control system of the SAV,may be downloadable from a remote storage database, and/or may bewirelessly accessible within the remote storage database in real-time.

To this end, FIG. 8 illustrates an exemplary Internet-based system,wherein SAVs are connected to a cloud-computing environment, and aremote terminal, via waypoints that are connected to the Internet.

According to an embodiment, an SAV 800 having a vehicle control system801 can connect to the Internet 880, via a wireless communication hub,through a wireless communication channel such as a base station 883(e.g., an Edge, 3G, 4G, or LTE Network), an access point 882 (e.g., afemto cell or Wi-Fi network), or a satellite connection 881. Acloud-computing controller 891 in concert with a cloud-computingprocessing center 892 can permit access to a data storage center 893.The data storage center 893 may contain a braking table database thatmay be accessed and/or downloaded by the SAV 800. The data storagecenter 893 may also be updated via a remote terminal 885. Thecloud-computing processing center 892 can be a computer duster, a datacenter, a main frame computer, or a server farm. In one implementation,the cloud-computing processing center 892 and data storage center 893are collocated.

In an embodiment, raw and/or processed information from a plurality ofvehicle sensors can be transmitted to the cloud-computing environment890 for processing by the cloud-computing processing center 892 and/orstorage in the data storage center 893. In the case of raw information,the cloud-computing processing center 892 can perform processing similarto that performed by the vehicle control system 801 of the SAV 800during SAV operation. These processes include, among other processes,object identification and image classification.

In an embodiment, the cloud-computing processing center 892 may be incommunication with a weather forecasting service (e.g., a meteorologicalservice) and may store, for transmittal to the SAV 800, weatherforecasts in accordance with a geographic location of the SAV 800, asdetermined by a Global Positioning System (GPS) receiver of the SAV 800.

According to an embodiment, a remote operator 886 can access thecloud-computing environment 890 through a remote terminal 885, such as adesktop or laptop computer or workstation that is connected to theInternet 880 via a wired network connection or a wireless networkconnection, in order to update braking tables that may be accessibleand/or downloadable by the SAV 800.

FIG. 9 is a block diagram of internal components of an example of avehicle control system (VCS) that may be implemented, according to anembodiment. As discussed above, the VCS may be an electronics controlunit (ECU). For instance, VCS 901 may represent an implementation of atelematics and GPS ECU or a video ECU. It should be noted that FIG. 9 ismeant only to provide a generalized illustration of various components,any or all of which may be utilized as appropriate. It can be notedthat, in some instances, components illustrated by FIG. 9 can belocalized to a single physical device and/or distributed among variousnetworked devices, which may be disposed at different physicallocations.

The VCS 901 is shown comprising hardware elements that can beelectrically coupled via a BUS 967 (or may otherwise be incommunication, as appropriate). The hardware elements may includeprocessing circuitry 961 which can include without limitation one ormore processors, one or more special-purpose processors (such as digitalsignal processing (DSP) chips, graphics acceleration processors,application specific integrated circuits (ASICs), and/or the like),and/or other processing structure or means. The above-describedprocessors can be specially-programmed to perform operations including,among others, image processing and data processing. Some embodiments mayhave a separate DSP 963, depending on desired functionality. The VCS 901also can include one or more input device controllers 970, which cancontrol without limitation an in-vehicle touch screen, a touch pad,microphone, button(s), dial(s), switch(es), and/or the like. The VCS 901can also include one or more output device controllers 962, which cancontrol without limitation a display, light emitting diode (LED),speakers, and/or the like.

The VCS 901 might also include a wireless communication hub 964, whichcan include without limitation a modem, a network card, an infraredcommunication device, a wireless communication device, and/or a chipset(such as a Bluetooth device, an IEEE 802.11 device, an IEEE 802.16.4device, a WiFi device, a WiMax device, cellular communication facilitiesincluding 4G, 5G, etc.), and/or the like. The wireless communication hub964 may permit data to be exchanged with, as described, in part, withreference to FIG. 8, a network, wireless access points, other computersystems, and/or any other electronic devices described herein. Thecommunication can be carried out via one or more wireless communicationantenna(s) 965 that send and/or receive wireless signals 966.

Depending on desired functionality, the wireless communication hub 964can include separate transceivers to communicate with base transceiverstations (e.g., base stations of a cellular network) and/or accesspoint(s). These different data networks can include various networktypes. Additionally, a Wireless Wide Area Network (WWAN) may be a CodeDivision Multiple Access (CDMA) network, a Time Division Multiple Access(TDMA) network, a Frequency Division Multiple Access (FDMA) network, anOrthogonal Frequency Division Multiple Access (OFDMA) network, a WiMax(IEEE 802.16), and so on. A CDMA network may implement one or more radioaccess technologies (RATs) such as cdma2000, Wideband-CDMA (W-CDMA), andso on. Cdma2000 includes IS-95, IS-2000, and/or IS-856 standards. A TDMAnetwork may implement Global System for Mobile Communications (GSM),Digital Advanced Mobile Phone System (D-AMPS), or some other RAT. AnOFDMA network may employ LTE, LTE Advanced, and so on, including 4G and5G technologies.

The VCS 901 can further include sensor controller(s) 974. Suchcontrollers can control, without limitation, the plurality of vehiclesensors 968, including, among others, one or more accelerometer(s),gyroscope(s), camera(s), RADAR(s), LiDAR(s), Ultrasonic sensor(s),magnetometer(s), altimeter(s), microphone(s), proximity sensor(s), lightsensor(s), and the like.

Embodiments of the VCS 901 may also include a Satellite PositioningSystem (SPS) receiver 971 capable of receiving signals 973 from one ormore SPS satellites using an SPS antenna 972. The SPS receiver 971 canextract a position of the device, using conventional techniques, fromsatellites of an SPS system, such as a global navigation satellitesystem (GNSS) (e.g., GPS), Galileo, Glonass, Compass, Quasi-ZenithSatellite System (QZSS) over Japan, Indian Regional NavigationalSatellite System (IRNSS) over India, Beidou over China, and/or the like.Moreover, the SPS receiver 971 can be used various augmentation systems(e.g., an Satellite Based Augmentation System (SBAS)) that may beassociated with or otherwise enabled for use with one or more globaland/or regional navigation satellite systems. By way of example but notlimitation, an SBAS may include an augmentation system(s) that providesintegrity information, differential corrections, etc., such as, e.g.,Wide Area Augmentation System (WAAS), European Geostationary NavigationOverlay Service (EGNOS), Multi-functional Satellite Augmentation System(MSAS), GPS Aided Geo Augmented Navigation or GPS and Geo AugmentedNavigation system (GAGAN), and/or the like. Thus, as used herein an SPSmay include any combination of one or more global and/or regionalnavigation satellite systems and/or augmentation systems, and SPSsignals may include SPS, SPS-like, and/or other signals associated withsuch one or more SPS.

In an embodiment, the SPS receiver 971 of the VCS 901 may be provided asa query to a weather forecasting service (e.g., a meteorologicalservice) in order to obtain a current weather condition in theenvironment surrounding the SAV. The query may be provided via directcommunication with a weather forecasting service via Internet and/or byaccessing a weather forecast stored and updated within a cloud-basedstorage center.

The VCS 901 may further include and/or be in communication with a memory969. The memory 969 can include, without limitation, local and/ornetwork accessible storage, a disk drive, a drive array, an opticalstorage device, a solid-state storage device, such as a random accessmemory (“RAM”), and/or a read-only memory (“ROM”), which can beprogrammable, flash-updateable, and/or the like. Such storage devicesmay be configured to implement any appropriate data stores, includingwithout limitation, various file systems, database structures, and/orthe like.

The memory 969 of the VCS 901 also can comprise software elements (notshown), including an operating system, device drivers, executablelibraries, and/or other code embedded in a computer-readable medium,such as one or more application programs, which may comprise computerprograms provided by various embodiments, and/or may be designed toimplement methods, and/or configure systems, provided by otherembodiments, as described herein. In an aspect, then, such code and/orinstructions can be used to configure and/or adapt a general purposecomputer (or other device) to perform one or more operations inaccordance with the described methods, thereby resulting in aspecial-purpose computer.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

With reference to the appended figures, components that can includememory can include non-transitory machine-readable media. The term“machine-readable medium” and “computer-readable medium” as used herein,refer to any storage medium that participates in providing data thatcauses a machine to operate in a specific fashion. In embodimentsprovided hereinabove, various machine-readable media might be involvedin providing instructions/code to processing units and/or otherdevice(s) for execution. Additionally or alternatively, themachine-readable media might be used to store and/or carry suchinstructions/code. In many implementations, a computer-readable mediumis a physical and/or tangible storage medium. Such a medium may takemany forms, including but not limited to, non-volatile media, volatilemedia, and transmission media. Common forms of computer-readable mediainclude, for example, magnetic and/or optical media, a RAM, a PROM,EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier waveas described hereinafter, or any other medium from which a computer canread instructions and/or code.

The methods, systems, and devices discussed herein are examples. Variousembodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, features described with respectto certain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner. The various components of the figures provided hereincan be embodied in hardware and/or software. Also, technology evolvesand, thus, many of the elements are examples that do not limit the scopeof the disclosure to those specific examples.

The methods, apparatuses, and devices discussed herein are examples.Various embodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, features described with respectto certain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner. The various components of the figures provided hereincan be embodied in hardware and/or software. Also, technology evolvesand, thus, many of the elements are examples that do not limit the scopeof the disclosure to those specific examples.

Obviously, numerous modifications and variations are possible in lightof the above teachings. It is therefore to be understood that within thescope of the appended claims, the invention may be practiced otherwisethan as specifically described herein.

Embodiments of the present disclosure may also be as set forth in thefollowing parentheticals.

(1) A method of a braking operation of a vehicle, comprising acquiringat least one image of an external environment of the vehicle,determining a road condition of a road of the external environment ofthe vehicle based on the acquired at least one image, obtaining, basedon the determined road condition and from memory, a braking table of oneor more braking tables including distances and corresponding vehiclespeeds at which the braking operation is performed, acquiring a speed ofthe vehicle and a distance between a preceding object and the vehicle,comparing the acquired speed of the vehicle and the acquired distancebetween the preceding object and the vehicle to the braking table, andsending, by processing circuitry and based on the comparison, aninstruction to perform the braking operation of the vehicle.

(2) The method according to (1), wherein the determining determines theroad condition by segmenting, within the acquired at least one image,the road of the external environment of the vehicle, and determining theroad condition based on the segmented road within the acquired at leastone image.

(3) The method according to either (1) or (2), wherein the determiningdetermines the road condition by classifying the segmented road asbelonging to one of a plurality of types of road conditions.

(4) The method according to any one of (1) to (3), wherein the pluralityof types of road conditions include ice, snow, rain, asphalt, cement,dirt, rocks, and combinations thereof.

(5) The method according to any one of (1) to (4), wherein the sendingincludes sending, by the processing circuitry, a first instruction toperform a pre-fill brake of a braking system of the vehicle when a firstcondition of the braking table is met, and sending, by the processingcircuitry, a second instruction to perform an emergency braking of thevehicle when a second condition of the braking table is met.

(6) The method according to any one of (1) to (5), wherein the firstinstruction to perform the pre-fill brake of the braking system of thevehicle includes pressurizing hydraulics of the braking system of thevehicle.

(7) The method according to any one of (1) to (6), wherein, during thepre-fill brake of the braking system of the vehicle, an acceleratorpedal of the vehicle is deactivated.

(8) The method according to any one of (1) to (7), wherein thedetermining determines the road condition based on the acquired at leastone image and by obtaining, based on a position of the vehicle,meteorological conditions of the external environment from anInternet-based meteorological service.

(9) An apparatus for providing a braking operation of a vehicle,comprising processing circuitry configured to acquire at least one imageof an external environment of the vehicle, determine a road condition ofa road of the external environment of the vehicle based on the acquiredat least one image, obtain, based on the determined road condition andfrom memory, a braking table of one or more braking tables includingdistances and corresponding vehicle speeds at which the brakingoperation is performed, acquire a speed of the vehicle and a distancebetween a preceding object and the vehicle, compare the acquired speedof the vehicle and the acquired distance between the preceding objectand the vehicle to the braking table, and send, based on the comparison,an instruction to perform the braking operation of the vehicle.

(10) The apparatus according to (9), wherein, to determine the roadcondition, the processing circuitry is further configured to segment,within the acquired at least one image, the road of the externalenvironment of the vehicle, and determine the road condition based onthe segmented road within the acquired at least one image.

(11) The apparatus according to either (9) or (10), wherein theprocessing circuitry is further configured to classify the segmentedroad as belonging to one of a plurality of types of road conditions.

(12) The apparatus according to any one of (9) to (11), wherein theplurality of types of road conditions include ice, snow, rain, asphalt,cement, dirt, rocks, and combinations thereof.

(13) The apparatus according to any one of (9) to (12), wherein, to sendthe instruction, the processing circuitry is further configured to senda first instruction to perform a pre-fill brake of a braking system ofthe vehicle when a first condition of the braking table is met, and senda second instruction to perform an emergency braking of the vehicle whena second condition of the braking table is met.

(14) The apparatus according to any one of (9) to (13), wherein, insending the first instruction, the processing circuitry is furtherconfigured to pressurize hydraulics of the braking system of thevehicle.

(15) A non-transitory computer-readable storage medium storingcomputer-readable instructions that, when executed by a computer, causethe computer to perform a method of a braking operation of a vehicle,the method comprising acquiring at least one image of an externalenvironment of the vehicle, determining a road condition of a road ofthe external environment of the vehicle based on the acquired at leastone image, obtaining, based on the determined road condition and frommemory, a braking table of one or more braking tables includingdistances and corresponding vehicle speeds at which the brakingoperation is performed, acquiring a speed of the vehicle and a distancebetween a preceding object and the vehicle, comparing the acquired speedof the vehicle and the acquired distance between the preceding objectand the vehicle to the braking table, and sending, by processingcircuitry and based on the comparison, an instruction to perform thebraking operation of the vehicle.

(16) The non-transitory computer-readable storage medium according to(15), wherein the determining determines the road condition bysegmenting, within the acquired at least one image, the road of theexternal environment of the vehicle, and determining the road conditionbased on the segmented road within the acquired at least one image.

(17) The non-transitory computer-readable storage medium according toeither (15) or (16), wherein the determining determines the roadcondition by classifying the segmented road as belonging to one of aplurality of types of road conditions.

(18) The non-transitory computer-readable storage medium according toany one of (15) to (17), wherein the plurality of types of roadconditions include ice, snow, rain, asphalt, cement, dirt, rocks, andcombinations thereof.

(19) The non-transitory computer-readable storage medium according toany one of (15) to (18), wherein the sending includes sending, by theprocessing circuitry, a first instruction to perform a pre-fill brake ofa braking system of the vehicle when a first condition of the brakingtable is met, and sending, by the processing circuitry, a secondinstruction to perform an emergency braking of the vehicle when a secondcondition of the braking table is met.

(20) The non-transitory computer-readable storage medium according toany one of (15) to (19), wherein the first instruction to perform thepre-fill brake of the braking system of the vehicle includespressurizing hydraulics of the braking system of the vehicle.

(21) The non-transitory computer-readable storage medium according toany one of (15) to (20), wherein, during the pre-fill brake of thebraking system of the vehicle, an accelerator pedal of the vehicle isdeactivated.

(22) The non-transitory computer-readable storage medium according toany one of (15) to (21), wherein the determining determines the roadcondition based on the acquired at least one image and by obtaining,based on a position of the vehicle, meteorological conditions of theexternal environment from an Internet-based meteorological service.

Thus, the foregoing discussion discloses and describes merely exemplaryembodiments of the present invention. As will be understood by thoseskilled in the art, the present invention may be embodied in otherspecific forms without departing from the spirit or essentialcharacteristics thereof. Accordingly, the disclosure of the presentinvention is intended to be illustrative, but not limiting of the scopeof the invention, as well as other claims. The disclosure, including anyreadily discernible variants of the teachings herein, defines, in part,the scope of the foregoing claim terminology such that no inventivesubject matter is dedicated to the public.

1. A method of a braking operation of a vehicle, comprising: acquiringat least one image of an external environment of the vehicle;determining a road condition of a road of the external environment ofthe vehicle based on the acquired at least one image; obtaining, basedon the determined road condition and from memory, a braking table of oneor more braking tables including distances and corresponding vehiclespeeds at which the braking operation is performed; acquiring a speed ofthe vehicle and a distance between a preceding object and the vehicle;comparing the acquired speed of the vehicle and the acquired distancebetween the preceding object and the vehicle to the braking table; andsending, by processing circuitry and based on the comparison, aninstruction to perform the braking operation of the vehicle.
 2. Themethod according to claim 1, wherein the determining determines the roadcondition by segmenting, within the acquired at least one image, theroad of the external environment of the vehicle, and determining theroad condition based on the segmented road within the acquired at leastone image.
 3. The method according to claim 2, wherein the determiningdetermines the road condition by classifying the segmented road asbelonging to one of a plurality of types of road conditions.
 4. Themethod according to claim 3, wherein the plurality of types of roadconditions include ice, snow, rain, asphalt, cement, dirt, rocks, andcombinations thereof.
 5. The method according to claim 1, wherein thesending includes sending, by the processing circuitry, a firstinstruction to perform a pre-fill brake of a braking system of thevehicle when a first condition of the braking table is met, and sending,by the processing circuitry, a second instruction to perform anemergency braking of the vehicle when a second condition of the brakingtable is met.
 6. The method according to claim 5, wherein the firstinstruction to perform the pre-fill brake of the braking system of thevehicle includes pressurizing hydraulics of the braking system of thevehicle.
 7. The method according to claim 6, wherein, during thepre-fill brake of the braking system of the vehicle, an acceleratorpedal of the vehicle is deactivated.
 8. The method according to claim 1,wherein the determining determines the road condition based on theacquired at least one image and by obtaining, based on a position of thevehicle, meteorological conditions of the external environment from anInternet-based meteorological service.
 9. An apparatus for providing abraking operation of a vehicle, comprising: processing circuitryconfigured to acquire at least one image of an external environment ofthe vehicle, determine a road condition of a road of the externalenvironment of the vehicle based on the acquired at least one image,obtain, based on the determined road condition and from memory, abraking table of one or more braking tables including distances andcorresponding vehicle speeds at which the braking operation isperformed, acquire a speed of the vehicle and a distance between apreceding object and the vehicle, compare the acquired speed of thevehicle and the acquired distance between the preceding object and thevehicle to the braking table, and send, based on the comparison, aninstruction to perform the braking operation of the vehicle.
 10. Theapparatus according to claim 9, wherein, to determine the roadcondition, the processing circuitry is further configured to segment,within the acquired at least one image, the road of the externalenvironment of the vehicle, and determine the road condition based onthe segmented road within the acquired at least one image.
 11. Theapparatus according to claim 10, wherein the processing circuitry isfurther configured to classify the segmented road as belonging to one ofa plurality of types of road conditions.
 12. The apparatus according toclaim 11, wherein the plurality of types of road conditions include ice,snow, rain, asphalt, cement, dirt, rocks, and combinations thereof. 13.The apparatus according to claim 9, wherein, to send the instruction,the processing circuitry is further configured to send a firstinstruction to perform a pre-fill brake of a braking system of thevehicle when a first condition of the braking table is met, and send asecond instruction to perform an emergency braking of the vehicle when asecond condition of the braking table is met.
 14. The apparatusaccording to claim 13, wherein, in sending the first instruction, theprocessing circuitry is further configured to pressurize hydraulics ofthe braking system of the vehicle.
 15. A non-transitorycomputer-readable storage medium storing computer-readable instructionsthat, when executed by a computer, cause the computer to perform amethod of a braking operation of a vehicle, the method comprising:acquiring at least one image of an external environment of the vehicle;determining a road condition of a road of the external environment ofthe vehicle based on the acquired at least one image; obtaining, basedon the determined road condition and from memory, a braking table of oneor more braking tables including distances and corresponding vehiclespeeds at which the braking operation is performed; acquiring a speed ofthe vehicle and a distance between a preceding object and the vehicle;comparing the acquired speed of the vehicle and the acquired distancebetween the preceding object and the vehicle to the braking table; andsending, by processing circuitry and based on the comparison, aninstruction to perform the braking operation of the vehicle.
 16. Thenon-transitory computer-readable storage medium according to claim 15,wherein the determining determines the road condition by segmenting,within the acquired at least one image, the road of the externalenvironment of the vehicle, and determining the road condition based onthe segmented road within the acquired at least one image.
 17. Thenon-transitory computer-readable storage medium according to claim 16,wherein the determining determines the road condition by classifying thesegmented road as belonging to one of a plurality of types of roadconditions.
 18. The non-transitory computer-readable storage mediumaccording to claim 17, wherein the plurality of types of road conditionsinclude ice, snow, rain, asphalt, cement, dirt, rocks, and combinationsthereof.
 19. The non-transitory computer-readable storage mediumaccording to claim 15, wherein the sending includes sending, by theprocessing circuitry, a first instruction to perform a pre-fill brake ofa braking system of the vehicle when a first condition of the brakingtable is met, and sending, by the processing circuitry, a secondinstruction to perform an emergency braking of the vehicle when a secondcondition of the braking table is met.
 20. The non-transitorycomputer-readable storage medium according to claim 19, wherein thefirst instruction to perform the pre-fill brake of the braking system ofthe vehicle includes pressurizing hydraulics of the braking system ofthe vehicle.